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Abstract 

Alumni self-ratings of their personal growth were linked to their intellectual 
development during college four to seven years earlier. Graduates that were 
satisfied with their personal growth in the arts, creative thinking, making logi¬ 
cal inferences, learning independently, exercising initiative, and tolerating other 
points of view had higher intellectual scores in Commitment and Empathy as 
undergraduates years earlier. These findings support a relationship between 
college student intellectual development and alumni perceptions of their per¬ 
sonal growth. The implications of this study support continuing the custom of 
querying graduates about their earlier education, a practice in wide use already; 
and add to the validity of the Scale of Intellectual Development as a measure of 

college impact upon personal dispositions. 
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M. he role of intellectual development is considered a central component of 
undergraduate education and receives much attention regarding its nature and impor¬ 
tance (Evans, Forney, Guido, Patton, & Renn, 2010; Pascarella & Terenzini, 2005). Many 
institutions track some elements of intellectual and cognitive development during col¬ 
lege, but what effects remain after college is also of fundamental interest. Separate arenas 
of research activity exist with current students and with alumni, but no longitudinal stud¬ 
ies link collegiate intellectual development to personal growth as perceived by alumni. 
Studying possible connections of collegiate student development with personal growth 
has implications for programming at the undergraduate level as well as our understanding 
of the lasting impact of development after college graduation. 

Intellectual Development 

Intellectual development occupies a key foundation in college impact studies 
(Collins, 2006; Dawson, 2004; Pascarella, 1985; Torres, 2003; Wang & Rodgers, 2006), 
and both academic and student affairs programs claim to nurture student’s progress in 
intellectual and cognitive development. A variety of conceptual approaches to cognitive 
development exist in the literature such as critical thinking, postformal reasoning, and 
dispositional critical thinking (Evans et al., 2010; Pascarella & Terenzini, 2005). Although 
critical thinking development is most often defined with cognitive skills such as “identify 
central issues and assumptions in an argument” (Jones et al., 1995, p. 133), the disposi¬ 
tion to think critically has also been noted as part of intellectual development. Some ex¬ 
amples of dispositional critical thinking include, “tolerance for new ideas” and “willingness 
to see complexity in problems” (Facione, Sanchez, Facione, & Gainen, 1995; Taube, 1997). 

Recently, the importance of intellectual development has come forth in the 
Lumina Foundation’s project called the Degree Qualifications Profile, which “defines 
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expected learning outcomes that graduates need for work, citizenship, global participation 
and life” (Adelman, Ewell, Gaston, & Schneider, 2011, p. 1). Partly motivated by Europe’s 
Bologna Process (2012), this Profile has outlined “broad, integrative knowledge” and “applied 
learning” skill sets in general education. One illustrative example of intellectual development 
states, “articulates and defends the significance and implications of his or her specialized 
work in terms of challenges, trends and developments in a social or global context” (p. 18). 
This expectation of intended learning outcomes has a developmental aspect that includes the 
importance of considering impact on other people and society in general. 

Some of our external constituents such as the business community propose affec¬ 
tive components are as important in employment and citizenship settings as the cognitive 
components (Jones et ah, 1995). This inclination to be open to evidence, to welcome new 
ideas, and to embrace complexity is as necessary or at least an equal partner to the pure 
cognitive component of reasoning skills (Facione et al., 1995). In some aspects of intel¬ 
lectual development, the cognitive and affective perspectives are inseparable. For example, 
tolerance has both cognitive and affective components: a rigidity of thinking and feeling 
(Erwin, 2000). It may be an artificial separation, but researchers (Facione et ah, 1995) have 
separated the dispositional perspective, and more positive dispositions are expected due to 
the undergraduate experiences. Pascarella and Terenzini (2005) report the need for further 
research in intellectual development in studying college impact, particularly using longitu¬ 
dinal designs. To address this need, in part, this study focuses on the dispositional aspects 
of intellectual development. 

Alumni Perceptions 

Colleges and universities have surveyed their alumni since the 1930’s (Pace, 1979), 
and the practice has grown in scope and consequence ever since. Information from alumni 
surveys is used to inform fundraising, college marketing, employment preparation, return 
on public investment, and institutional accountability such as accreditation, program re¬ 
view, and performance funding (Allen, Ramaekers, & van der Velden, 2005; Borden, 2005; 
Ewell, 2005; Volkwein, 2010). Over a dozen states conduct common comprehensive alumni 
surveys (Ewell, 2005), and several countries have incorporated the practice as well (Weerts 
& Vidal, 2005). 

In spite of the ubiquitous nature of alumni surveys, questions are still posed about 
the credibility of self-report data (Bowman & Seifert, 2011; Pike, 2011). Nevertheless, as 
the foci of surveys have expanded from employment history to college outcomes, it is wise 
to keep in mind some possible limitations of survey methodologies as well as continue 
research into their validity. Some of the issues raised about survey methodology include 
the potential limits of generalizability due to low response rates, the possibility of other life 
events being confused with college impact, and the role of “indirect” measures versus “di¬ 
rect” measures of student development. In spite of these concerns, alumni perceptual data 
maintains a prominent place in institutional assessment portfolios. 

Given the place of alumni information in institutions, it follows that information 
obtained during the undergraduate years be studied for its value in optimizing alumni 
perceptions. Before embarking on research studies with alumni, Volkwein (2010) recom¬ 
mends that researchers adopt an intended perspective. This study utilized the perspective 
of a developmental outcome model. That is, an intellectual developmental model based 
on dispositions was considered as the underlying expectation or educational objective for 
undergraduate impact. Stated another way, higher intellectual development should occur 
because of college. Generally, this study determines if average differences in college intel¬ 
lectual development occur between alumni satisfied or dissatisfied in nine areas of personal 
growth. These areas of personal growth are: the arts, creative thinking, making logical 
inferences based on assumptions, independent learning, setting personal goals, self-reliance, 
exercising initiative, persistence, and tolerating other points of view. It was hypothesized that 
satisfied alumni in their personal growth had higher intellectual development during college. 



Method 
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Data Source and Participants 

Samples of late sophomores (45-70 semester hours completed) enrolled during 
1995-2000 at a moderately selective mid-sized institution in the mid-Atlantic area were 
selected as participants for this study. Students were chosen if the last digit of their identi¬ 
fication number ended in seven. This sample was considered as quasi-random because no 
selection bias was associated with the last digit. Students in this sample were given a devel¬ 
opmental test in a proctored setting. Although no comparison was made with other student 
groups (i.e. students whose identification did not end in seven) who completed other as¬ 
sessment instruments, no motivation bias was believed to be present with administration of 
this particular instrument to students with an id digit of seven. 

Additional datasets were also obtained from institutional alumni surveys admin¬ 
istered between 2002 and 2008. As part of the institution’s ongoing assessment process, 
alumni surveys are administered every year to alumni who had graduated from a given 
group of major programs. Major programs participate in alumni surveys every six years, and 
graduates from all major programs at the institution are surveyed within five years. Insti¬ 
tutionally, all graduates are contacted within one to five years after program completion. 
Approximately 40-50% of the graduates contacted responded to the survey, resulting in a 
sample matched with the Scale of Intellectual Development of complete data for this study 
of n=624 (See Table 1). 

Table 1 


Frequencies in Longitudinal Design of Late Sophomores Completing the Scale of Intellectual 
Development by Year of Alumni Survey Administration 


Alumni Survey 

1995 

Scale of Intellectual Development 

1996 1997 1998 1999 

2000 

Subtotals 

2002 

47 

0 

0 

0 

0 

0 

47 

2003 

63 

81 

4 

0 

0 

0 

148 

2004 

38 

86 

22 

0 

0 

0 

146 

2005 

29 

28 

22 

18 

36 

0 

133 

2006 

0 

0 

1 

0 

6 

5 

12 

2007 

0 

0 

21 

12 

26 

22 

81 

2008 

0 

0 

0 

3 

26 

28 

57 

Totals 

177 

195 

70 

33 

94 

55 

624 


Instruments 

The Scale of Intellectual Development (SID) (Erwin, 1983) was administered to 
a sample of sophomores who had completed 45-70 semester hours. Originally based on 
Perry’s (1968) scheme of intellectual development , the SID has four subscales: Dualism, 
Relativism, Commitment, and Empathy. For the Dualism subscale, individuals scoring high 
on this factor tend to see issues in clear-cut, categorical terms, and look to authorities for 
the answers. For the Relativism subscale, individuals scoring high on this factor recognize 
alternative perspectives exist and can handle diversity within themselves and in relation to 
other people; but they still believe outside forces determine their future. For the Commit¬ 
ment subscale, individuals scoring high on this factor have begun to make major decisions 
in their lives and accept the responsibilities and consequences of these decisions. There is 
a tolerance of other viewpoints; however, these persons know where they stand in relation 
to other people. For the Empathy subscale, individuals scoring high on this factor not only 
have made major life decisions but also are aware of their impact on other people. High 
scorers have developed sensitivity about other people and feel responsible for improving so¬ 
ciety in general. Validity evidence for the SID as a measure of intellectual development may 
be found in Buczynski (1994), DeMars and Erwin (2003), Erwin (1993), and Erwin (2000). 
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Only Commitment and Empathy SID sub-scales were used in this study because these de¬ 
velopmental dimensions are at the higher end of intellectual development and expected of 
college graduates. In addition, Commitment and Empathy were hypothesized to be related 
to the areas of personal growth collected from graduates. For the current sample, the sub¬ 
scales of Commitment and Empathy had Cronbach’s alpha internal consistency coefficients 
of .80 and .86, respectively. 

On the alumni survey, graduates were asked to indicate their employment and con¬ 
tinuing education history; overall satisfaction with the institution, major program, and vari¬ 
ous student services; satisfaction with several academic areas such as writing, mathemat¬ 
ics, speaking, and technology; and satisfaction with the institution’s contribution to their 
personal growth in nine areas: the arts, creative thinking, making logical inferences based 
on assumptions, independent learning, selection of personal goals, self-reliance, exercising 
initiative, persistence, and tolerating other points of view. Alumni responded on a 4-point 
Likert scale ranging from very dissatisfied to very satisfied for each growth area. Responses 
were categorized in this study into either satisfied or dissatisfied for easier interpretability. 

Procedure 


Some of our external 
constituents such 
as the business 
community propose 
affective components 
are as important in 
employment and 
citizenship settings as the 
cognitive components. 


The longitudinal sample in this study consisted of students who had completed the 
SID and later an alumni survey. The time between late sophomore developmental scores 
and post-graduate perceptions ranged from four to six years. The reason for this span was 
due to years spent until graduation and the target years the alumni survey was adminis¬ 
tered. Table 1 lists the frequencies related to year of cognitive development testing with 
year of completing alumni survey. 

Multivariate analysis of variance (MANOVA) statistics were calculated using each 
of the nine alumni personal growth areas as independent variables and two subscales 
from the SID scale, Commitment and Empathy, as the dependent variables. MANOVA was 
used to study average differences in intellectual development assessed during college for 
satisfied or dissatisfied alumni in several areas of personal growth. The Commitment and 
Empathy subscales correlated 0.65; therefore, Roy’s Maximum Root Criterion was chosen 
as the MANOVA statistic because it is most powerful when the dependent variables are con¬ 
centrated in a single variate (Hair, Black, Babin, & Anderson, 2010). Cohen’s d (Ilartung, 
Knapp, & Sinha, 2008) was the effect size calculated in this study; d essentially reports the 
magnitude between groups in standard deviation units. Typically effect sizes of .2 or lower 
are considered small, .5 moderate, and .8 large differences. Cohen has noted that effect 
sizes in personal and social areas are likely to be smaller than in achievement tests. 


Results 


Table 2 reports the sophomore Commitment and Empathy means, Cohen’s d sta¬ 
tistics, and F statistics between the satisfied and dissatisfied alumni. MANOVA statistics are 
reported for each independent variable below. 


Table 2 

Means, Cohen’s d, and Follow-Up ANOVA Statistics 


SID Subscale F Satisfied (n) Dissatisfied («) Cohen’s d F Satisfied («) Dissatisfied («) Cohen’s d 


Appreciation of Arts 

7.47** 

62.6 (482) 

54.80 (74) 

0.33 

17.57*** 

57.25 (482) 

49.81 (74) 

0.51 

Creative Thinking 

5.61** 

62.43 (481) 

55.58 (75) 

0.29 

7.13** 

56.50 (481) 

52.13 (75) 

0.33 

Logical Inferences 

5.61** 

62.39 (492) 

55.00 (63) 

0.31 

9.78*** 

56.89 (492) 

50.95 (63) 

0.42 

Learning Independently 

8.17** 

62.62 (486) 

54.01 (68) 

0.37 

9.26** 

56.92 (486) 

51.33 (68) 

0.39 

Personal Goals 

3.02 

62.11 (506) 

56.04 (49) 

0.26 

5.42* 

56.72 (506) 

51.77(40) 

0.35 

Initiative 

5.54** 

62.28 (507) 

54.00 (48) 

0.30 

11.06** 

56.85 (507) 

49.75 (48) 

0.50 

Other Points of View 

7.28** 

62.58 (446) 

55.81(105) 

0.29 

9.09*** 

56.91 (446) 

52.30 (105) 

0.33 


*p<.05, **p<.01, ***/?<.001 
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In general, graduates “satisfied” with their education in the arts, creative thinking, logical 
inferences, learning independently, exercising initiative, and tolerating other points of view 
had higher Commitment and Empathy scores as sophomores than “dissatisfied” graduates 
in these areas of personal growth. Graduates satisfied with their ability to set personal goals 
had higher average Empathy scores than dissatisfied graduates. No differences were found 
in either sophomore Commitment or Empathy scores for the personal growth areas of self- 
reliance or persistence. 

Also reported in Table 2 are Cohen’s d statistics across the various personal growth 
areas. These were small, ranging from a low of 0.29 for Commitment in creative thinking to 
a high of 0.51 for Empathy in the arts. 

Appreciation of Arts 

The MANOVA overall statistic of Roy’s Maximum Root Criterion was 0.03, F{ 2, 

555)=9.22,^>=.0001 for the alumni survey question about developing appreciation in the 
arts. Analysis of variance follow-up results for Commitment was .F(l, 556 =7.47,^>=.006; 
and for Empathy was F( 1, 556)=17.57, p=.0001. “Satisfied” Arts Commitment mean of 

62.78 (n = 484) was greater than the “dissatisfied” Commitment mean of 54.81 (n=74). 

The “satisfied” Arts Empathy mean of 57.25 (n = 484) was greater than the “dissatisfied” 

Empathy mean of 49.81 (n = 74). 

Creative Thinking 

Roy’s Maximum Root Criterion was 0.01, F(2, 555) = 3.79, p = .02 for the alumni 
question pertaining to creative thinking. Analysis of variance follow-up results for Commit¬ 
ment was F( 1, 556) = 5.85, p = .01; and Empathy was F(l, 556) = 7.42, p = .006. Gradu¬ 
ates “satisfied” with creative thinking had a Commitment mean 62.61 (n = 483), which was 
greater than “dissatisfied” Commitment mean of 55.58. Graduates “satisfied” with their 
creative thinking had a higher Empathy average of 56.96 (n = 483) than the “dissatisfied” 
average of 52.13 (n = 75). 

Logical Inference 

Roy’s Maximum Root Criterion was 0.01, F( 2, 554) = 5.03, p = .006 for the alumni 
survey question about making logical inferences. ANOVA follow-up for Commitment was 
F( 1, 555) = 5.83, p = .01; and Empathy was F{ 1, 555) = 10.06,^ = .001. Graduates “satis¬ 
fied” with their education in developing logical inference had Commitment means of 62.56 
(n = 494), which was greater than “dissatisfied” alumni having a mean of 55.00 (n = 63). 

Learning Independently 

Roy’s Maximum Root Criterion was 0.01, F( 2, 553) = 5.04, p = .006 for the alumni 
survey question about Learning Independently. ANOVA follow-up for Commitment was 
F( 1, 554) = 8.43,^ = .003; and Empathy was F( 1, 554) = 9.56 ,p = .002. Commitment 
mean for graduates “satisfied” with their education about Learning Independently was 

62.79 (n = 488), which was greater than the mean of “dissatisfied” alumni of 54.01 (n = 68). 

Empathy mean for “satisfied” alumni was 57.0 (n = 488), which was higher than the “dis¬ 
satisfied” mean of 51.33 (n = 68). 

Personal Goals 

Roy’s Maximum Root Criterion was 0.01, F( 2, 554) = 4.96, p = .05. ANOVA follow¬ 
up for Commitment was F{ 1, 555) = 3.16,^ = .07; and Empathy was F( 1, 555) = 5.61, 
p = 0.01. No statistical difference was found on Commitment between graduates “satisfied” 
with their education about their Personal Goal development and “dissatisfied” alumni. The 
Empathy mean for “satisfied” alumni was 56.80 (n = 508), which was higher than the “dis¬ 
satisfied” mean of 51.78 (n = 49). 
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Self-Reliance 

Roy’s Maximum Root Criterion was 0.009, F(2, 555) = 2.55,^=.07 for the alumni 
question pertaining to Developing Self-Reliance. No follow-up ANOVA were reviewed because 
this overall MANOVA p value of .07 was greater than the preset alpha level of .01. Therefore, 
no differences were found on Commitment or Empathy between alumni “satisfied” versus 
“dissatisfied” with their prior collegiate experience in Developing Self-reliance. 

Initiative 

Roy’s Maximum Root Criterion was 0.02, F( 2, 554) = 5.74, p = .003. ANOVA follow¬ 
up for Commitment was F{ 1, 555) = 5.72 ,p = .01; and Empathy was F{ 1, 555) = 11.29, 
p = .0008. Commitment mean for graduates “satisfied” with their education developing 
their capacity for Exercising Initiative was 62.45 (n = 509), which was greater than the 
mean of “dissatisfied” alumni of 54.00 (n = 48). Empathy mean for “satisfied” alumni was 
56.96 (n = 509), which was higher than the “dissatisfied” mean of 49.75 (n = 48). 

Persistence 

Roy’s Maximum Root Criterion was 0.004, F( 2, 555) = 1.25, p = .28. This p value 
of .28 was greater than the preset alpha of .01, and no follow-up ANOVA were reviewed. No 
differences were found between alumni “satisfied” versus “dissatisfied” in their prior col¬ 
legiate experience of Developing Persistence on either Commitment or Empathy. 

Tolerating Other Points of View 

Roy’s Maximum Root Criterion was 0.01, F{2, 550) = 4.89, p = .007. ANOVA follow¬ 
up for Commitment wasEfl, 551) = 7.62, p = .006; and Empathy wasE(l, 551) = 9.50, 
p = .002. Commitment mean for graduates “satisfied” with their education for Tolerating 
Other Points of View was 62.76 (n = 448), which was greater than the mean of “dissatis¬ 
fied” alumni of 55.82 (n = 105). Empathy mean for “satisfied” alumni was 57.05 (n = 448), 
which was higher than the “dissatisfied” mean of 52.03 (n = 448). 

Discussion 

Graduates who responded to an alumni survey about their satisfaction with sev¬ 
eral areas of personal growth were linked with their sophomore intellectual development 
scores, as measured by the SID, four to six years prior. Unlike most previous studies that 
focus either on current students or alumni, this study examines student development over 
time. “Satisfied” graduates in their abilities with the arts, creative thinking, logical infer¬ 
ences, learning independently, exercising of initiative, and tolerating other points of view 
had higher Commitment and Empathy scores on the average than “dissatisfied” graduates 
in these areas of personal growth. Graduates satisfied with their education in personal goals 
had higher Empathy average scores than dissatisfied graduates. No differences in intellectu¬ 
al development were found between “satisfied” and “dissatisfied” graduates in self-reliance 
or developing persistence. Personal goals had mixed results; no differences were found for 
Commitment, but satisfied graduates had higher Empathy scores. While it is not known 
why differences in intellectual development were not found in these areas, greater intellec¬ 
tual development in college may not be expected to be related to all these areas of personal 
growth. Self-reliance and persistence may be related to other areas of college student devel¬ 
opment but not intellectual development. 

These results are limited to graduates who responded to the institutional alumni 
survey and who had completed the SID, which was administered to quasi-random samples 
of late sophomores four to six years earlier. Approximately half of the graduates who had 
taken the intellectual development test responded to the alumni survey. It is not known 
if the non-responding group of alumni would change these results had they responded. 
“Satisfaction” was defined and reported by the graduates themselves that might vary by 
self-referencing differences on the personal growth dimensions, and by a greater proportion 



• RESEARCH if PRACTICE IN ASSESSMENT 


of graduates who might acquiesce or be “satisfied” with institution. Despite the average dif¬ 
ferences found and reported in several personal growth areas, the effect sizes using Cohen’s 
d were moderate. Some of the differences may be due to larger sample sizes, and future 
studies should continue to monitor effect size. On the other hand, the effect sizes in this 
study are similar to freshmen to senior differences reported in the low .30’s for the major 
proprietary general education tests of Collegiate Learning Assessment, the Collegiate As¬ 
sessment of Academic Proficiency, and the Proficiency Profile (Klein et al., 2009). 

Despite these possible limitations, this study offers the strength of a longitudinal 
study and the collection of a “direct” measure of intellectual development. Although this 
study did not seek to identify what variables were associated with higher or lower intellectual 
development scores during college, prior research (e.g. Pascarella & Terenzini, 2005) has 
shown that offering academic and student services varying in perspective and opinions helps 
students reach commitments. In addition, students’ empathy can be enhanced by having 
them consider the effects of their and others’ decisions on society and humankind in general. 
Both Commitment and Empathy are currently supported in concept in the Degree Qualifica¬ 
tions Profile. Whatever the reasons for students being higher in Commitment and Empathy 
as undergraduates, graduates satisfied with several personal growth dimensions are associ¬ 
ated with higher commitment and empathy scores from their earlier college years. 

This study also supports the importance of intellectual development on graduates’ 
perception about their own collegiate experience. The concept of general education and ge¬ 
neric intellectual development skills is challenged by voices wishing to shorten the length 
of the undergraduate period either by eliminating general education entirely or to focus 
just on the major or professional education. This study also supports the value of graduates’ 
self-ratings of several areas of personal growth; satisfied graduates also had higher intel¬ 
lectual development scores in their college years. Often alumni perceptions are questioned 
as to their value: are they isolated or are they important as revelations to earlier collegiate 
development? Causal links are not made here, but relationships between alumni growth 
with earlier intellectual development was encouraging. This relationship supports college 
impact and value of alumni perceptions. 

Future studies may utilize other conceptualizations of intellectual development 
and other alumni personal growth areas. Retesting in intellectual development during the 
undergraduate period and also in direct testing of alumni in intellectual development may 
also enhance, or cast other perspectives on these findings. Other areas of student develop¬ 
ment besides intellectual development may be used such as moral or psychosocial develop¬ 
ment. Using more content-related measures of general education such as scientific reason¬ 
ing or communication abilities might be examined too. The undergraduate experience is 
complex, and additional dimensions of general education would be useful to study. 

Nevertheless, this study encourages future work in intellectual development during 
the collegiate experience and in continued reliance on alumni perceptions. Longitudinal 
studies of this type are also desirable given the current emphasis on lifelong learning. The 
link of association in this study supports further activities in both areas of intellectual de¬ 
velopment and in alumni information. 
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